Skip to content

Add op-attribution profiling for AgentX points - #3639

Draft
hbarclay wants to merge 14 commits into
mainfrom
hbarclay/profiling
Draft

Add op-attribution profiling for AgentX points#3639
hbarclay wants to merge 14 commits into
mainfrom
hbarclay/profiling

Conversation

@hbarclay

@hbarclay hbarclay commented Oct 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Opt-in op-attribution profiling for single-node AgentX points on vLLM, SGLang and TRT-LLM (NVIDIA and AMD). Set profile on e2e-tests.yml ("{}" for defaults); without it nothing changes.

Per run:

  • Every kernel, memcpy and memset with its op, input shapes, module stack and Python launcher, including CUDA/HIP graph replays.
  • Per-kernel clocks, throttle reasons and power; per-step energy.
  • Per-step batch composition, phase, graph mode and MoE routing.
  • A prefill window and a steady-decode window on every rank.

Artifacts: profile_<result> (raw traces) and profile_steps_<result> (per-step JSON, index.json, report.json). Details: inferencex-e2e/benchmarks/profiling/README.md.

🤖 Generated with Claude Code

@hbarclay hbarclay changed the title Op-attribution profiling for single-node vLLM runs: per-kernel ops, modules, clocks and MoE routing Single-node vLLM op-attribution profiling: per-kernel op/module, clocks, MoE routing Oct 1, 2026
@hbarclay hbarclay changed the title Single-node vLLM op-attribution profiling: per-kernel op/module, clocks, MoE routing Single-node vLLM op-attribution profiling (NVIDIA + AMD): per-kernel op/module, clocks, MoE routing Oct 5, 2026
@hbarclay hbarclay changed the title Single-node vLLM op-attribution profiling (NVIDIA + AMD): per-kernel op/module, clocks, MoE routing Single-node op-attribution profiling for vLLM, SGLang and TRT-LLM (NVIDIA + AMD): per-kernel op/module, clocks, MoE routing Oct 14, 2026
@hbarclay @claude
Add op-attribution profiling for AgentX points
59d4298 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@hbarclay hbarclay changed the title Single-node op-attribution profiling for vLLM, SGLang and TRT-LLM (NVIDIA + AMD): per-kernel op/module, clocks, MoE routing Add op-attribution profiling for AgentX points Oct 16, 2026
hbarclay and others added 13 commits October 16, 2026 19:58
@hbarclay @claude
Report unattributed activity placed in steps
5361713 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@hbarclay @claude
Mark SGLang deduplicated CUDA graph replays
26c2525 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@hbarclay @claude
Capture Muse Spark routing from its router
8fdfcd6 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@hbarclay @claude
Reset routing per step; sum token-sharded routing across ranks
2358b19 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@hbarclay @claude
Join graph replays with launch-less peer copy nodes
ae7b785 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@hbarclay @claude
Add engine_args profile setting and AMD SMI CLI clock fallback
b20b437 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@hbarclay @claude
Merge main
f02cd8c 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@hbarclay @claude
Add Muse Spark B300 TP8 HiCache c1 and c4
ad6f274 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@hbarclay @claude
Wait for trace exports before stopping the load; skip truncated traces
c1daa00 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@hbarclay @claude
Add Kimi-K3 B300 DSpark GPU-resident KV recipe
b61c16d 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@hbarclay @claude
Size the vLLM profiler to the longest window
9e8d957 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@hbarclay @claude
Issue synthetic load in waves
a99b4e2 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@hbarclay @claude
Open the synthetic decode window when every request in a wave is deco…
73f37da 
…ding

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Ankur-singh Ankur-singh Awaiting requested review from Ankur-singh Ankur-singh will be requested when the pull request is marked ready for review Ankur-singh is a code owner
@kedarpotdar-nv kedarpotdar-nv Awaiting requested review from kedarpotdar-nv kedarpotdar-nv will be requested when the pull request is marked ready for review kedarpotdar-nv is a code owner
@wzhao18 wzhao18 Awaiting requested review from wzhao18 wzhao18 will be requested when the pull request is marked ready for review wzhao18 is a code owner
@nvpohanh nvpohanh Awaiting requested review from nvpohanh nvpohanh will be requested when the pull request is marked ready for review nvpohanh is a code owner
@RohitNagraj RohitNagraj Awaiting requested review from RohitNagraj RohitNagraj will be requested when the pull request is marked ready for review RohitNagraj is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

InferenceX Board
Status: No status

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@hbarclay